2.1 Filtering out participants

# Loading the data to filter out participants
dataExp1 <- read.delim("./dataNotFilteredExp1.txt", sep ="\t", header = TRUE)
# Remove one participant who did 192 items instead of 303
nbResponse <- aggregate(dataExp1$enteredResponse, by=list(dataExp1$workerId),  FUN=length)
rmParticipant1Exp1 <- nbResponse[!nbResponse$x %in% c(303),]$Group.1 # CEA6BA7F432A7DD9F3352BD5AF605FED
dataExp1 <- dataExp1[!dataExp1$workerId %in% rmParticipant1Exp1,]
# Remove three participants who gave ratings lower than 4 to 3 most common Māori words borrowed into New Zealand English
list_carrot <- c("haka","kai","aotearoa")
carrots <- dataExp1[dataExp1$word %in% list_carrot,]
carrots <- carrots[as.numeric(carrots$enteredResponse) < 4,]
rmParticipant2Exp1 <- unique(carrots$workerId) # C2172ACC5CE7A076DF4AD48672826D17 CF5B1FC9D9FDDE28DABFB8B10F60F2A8 F317F5E5817BC9CA03FFA3D0AEE15BCC
dataExp1 <- dataExp1[!dataExp1$workerId %in% rmParticipant2Exp1,]
# Remove ratings for the three words (used to detect outliers)
dataExp1 <- dataExp1[!dataExp1$word %in% c("haka","kai","aotearoa"),]
# Remove two participants whose speakMaori or compMaori is at least (equal to or above) 3
rmParticipant3Exp1 <- unique(dataExp1[dataExp1$speakMaori >= 3 | dataExp1$compMaori >= 3,]$workerId)  # 0CAEF0D07D102126BADE4B5F353F3549 FFF87DE6A3C8DCD73B120BD06C3EB966
dataExp1 <- dataExp1[!dataExp1$workerId %in% rmParticipant3Exp1,]
# Remove one participant who did not learn their English in NZ and have been living in their current location in NZ for less than ten years (duration == "short")
summaryExp1WorkerId <- unique(dataExp1[,c("workerId","firstLangCountry","place","duration")])
EngNotInNZExp1 <- summaryExp1WorkerId[!summaryExp1WorkerId$firstLangCountry=="NZ",]
rmParticipant4Exp1 <- unique(EngNotInNZExp1[EngNotInNZExp1$duration=="short",]$workerId) # 31F2039C03219AE69C49AC468F37CA78
dataExp1 <- dataExp1[!dataExp1$workerId %in% rmParticipant4Exp1,]
# Remove two participants who know any other Polynesian languages
rmParticipant5Exp1 <- unique(dataExp1[dataExp1$anyPolynesian=="Yes",]$workerId) # 2B474DC72567F38FD065757F11211DFD 2B92687E2CE8ABD8047732400DFDC424
dataExp1 <- dataExp1[!dataExp1$workerId %in% rmParticipant5Exp1,]
# Remove five participants with language impairments
rmParticipant6Exp1 <- unique(dataExp1[dataExp1$impairments=="Yes",]$workerId) 
# 1DD4DB335A0E2735A9AE2AD898AAEFD5 2C40866C3268DB45274C535937D3DA24 933DFF677472628574F9DB2874121FD9 D42298BA8985A7C8E6232CCFC8E99A31 F4599D280C20128B3F6E804C0D2726B0
dataExp1 <- dataExp1[!dataExp1$workerId %in% rmParticipant6Exp1,]
# Remove one participant whose pattern of responses (SD) is below 2SD of the mean of all participants
SD <- aggregate(dataExp1$enteredResponse, by=list(dataExp1$workerId), sd)
cut <- mean(SD$x)-2*sd(SD$x)
rmParticipant7Exp1 <- SD[!SD$x > cut,]$Group.1 # B855CE4A15710AA41194B3D81EE68187
dataExp1 <- dataExp1[!dataExp1$workerId %in% rmParticipant7Exp1,]
# Check the total number of usable participants for Exp1
# length(unique(dataExp1$workerId)) # 85

2.2 Dataset structure

The data is structured as follows:

• workerId is the unique ID for each participant.
• enteredResponse is the wellformedness rating for each stimulus.
• reactionTime is the reaction time for each rating (second).
• type is the classification of each stimulus: nonword or word.
• length is the phoneme length of each stimulus.
• word is the stimulus used for the rating.
• speakMaori is each participant’s answer to the question how well they can speak Māori (with a scale ranging from 0 to 5).
• compMaori is each participant’s answer to the question how well they can understand/read Māori (with a scale ranging from 0 to 5).
• maoriProf is the sum of quantified response for speakMaori and compMaori, which refers to the level of each participant’s Māori proficiency.
• age is the age group where each participant belongs to .
• gender is the gender of each participant.
• ethnicity is categorized into binary answers, either Māori (M) or non Māori (non M).
• education is each participant’s highest level of education.
• children is each participant’s answer to the question whether they have had any children who have attended preschool or primary school in New Zealand in the past five years.
• maoriList is each participant’s basic knowledge of Māori (with a scale ranging from 0 to 9).
• place is each participant’s current place of living (categorized into binary classification, either North or South Island in New Zealand).
• duration is each participant’s time duration of living in their current place (categorized into binary classification, long: > 10 years and short: =< 10 years).
• firstLang is each participant’s first language.
• firstLangCountry is the country where each participant learned their first language.
  
• anyOtherLangs is the information regarding any other languages each participant can speak.
• hawaii is the binary response to the question whether participants have lived in Hawaii.
• anyPolynesian is the binary response to the question whether participants know any Polynesian such as Hawaiian, Tahitian, Sāmoan, or Tongan.
• whichPolynesian is the information regarding participants’ knowledge of any Polynesian languages if they knew any.
• impairments is the answer to the question whether participants have a history of any speech or language impairments.
• maoriExpo is each participant’s level of exposure to Māori (with a scale ranging from 0 to 10).
• Freq is the frequency of real word stimulus obtained from Māori running speech corpora (Freq=0 for pseudowords).
• bin is the classification of each stimulus according to their frequency of occurence in Māori running speech corpora.
• score is the dictionary phonotactic score normalized by the phonemic length of each stimulus.

2.3 Overview of participants’ sociolinguistic profile

Fig.S1: Overview of participants’ sociolinguistic profile in Exp1.

2.4 Modeling NMS’ confidence ratings in all bins with a linear mixed-effects model

# Model for TableS1
# modelTableS1 <- lmer(enteredResponse ~ c.(length)*c.(score) + type*bin + (1 + c.(length)*c.(score) + type*bin|workerId) + (1|word), control=lmerControl(optimizer="bobyqa"), dataExp1)
# saveRDS(modelTableS1, file = "modelTableS1.rds")
modelTableS1 <- readRDS("./modelTableS1.rds")
kable(xtable(summary(modelTableS1)$coef), digits=3, caption="Table.S1: Modeling NMS' confidence ratings in all bins with a linear mixed-effects model. All numeric variables in this model are centered.")

2.5 Effect plots from the model presented in Table.S1

Fig.S2: Effect plots of the interaction between bin and the distinction between nonwords and real words (Fig.S2a, on the left) and the interaction between phonotactic score and length of stimuli (Fig.S2b, on the right). The x-axis represents the type of bins (Fig.S2a) or the range of phonotactic score (Fig.S2b).

2.6 Code for plotting Fig.1

Fig.1: Mean rating for each stimulus per bin with predicted rating for each density plot. Bin1 contains the most frequent words and Bin5 consists of the least frequent words.

2.7 Mean rating and phonotactic score for each stimulus per bin

Fig.S3: Mean rating and phonotactic score for each stimulus per bin. The range of mean rating is represented on the y-axis and the x-axis represents the range of phonotactic score.

2.8 Rematching stimuli pairs

2.8.1 Rematched pairs: Mean rating and phonotactic score for each stimulus per bin

Fig.S4: Mean rating and phonotactic score for each stimulus per bin (rematched pairs). The range of mean rating is represented on the y-axis and the x-axis represents the range of phonotactic score.

During the analysis of Exp1, we found an overall bias towards higher phonotactic score of Māori words and lower phonotactic score of Māori-like nonwords due to a lack of nonwords to phonotactically match randomly selected words from Māori RS corpora. To solve this problem, nonwords and words are rematched across participants by finding a phonotactically matched word with the lowest absolute difference for each nonword. Among the rematched pairs, only those whose absolute difference is below 0.3 are kept. In the initial dataset, there are 200 pairs in each bin but after rematching stimuli, the number of pairs in each bin is reduced as follows: 71 (Bin1), 52 (Bin2 and Bin3), 53 (Bin4), 56 (Bin5).

# Model for Table.S2
# modelTableS2 <- lmer(enteredResponse ~ type + (1 + type|workerId) + (1|word), control=lmerControl(optimizer="bobyqa"), dataRematched) # saveRDS(modelTableS2, file = "modelTableS2.rds")
modelTableS2 <- readRDS("./modelTableS2.rds")
kable(xtable(summary(modelTableS2)$coef), digits=3, caption="Table.S2: Modeling NMS' confidence ratings with a linear mixed-effects model (rematched pairs).")

3.1 Filtering out participants

# Loading the data to filter out participants:
dataNotFiltered <- read.delim("./dataNotFilteredExp2.txt", sep ="\t", header = TRUE)
# Removing unusable MS participants
dataMS <- dataNotFiltered[dataNotFiltered$group=="MS",]
# Remove one participant who did 285 items instead of 320 (length 5)
nbResponse <- aggregate(dataMS$enteredResponse, by=list(dataMS$workerId),  FUN=length)
rmParticipant1 <- nbResponse[!nbResponse$x %in% c(320,240),]$Group.1 #634726494EE0F397CE0F525B97FA6B76
dataMS <- dataMS[!dataMS$workerId %in% rmParticipant1,]
# Remove two participants whose speakMaori or compMaori is below 3
rmParticipant2 <- unique(dataMS[dataMS$speakMaori <3 | dataMS$compMaori < 3,]$workerId) 
#83D25CDFA45FC1A34F4A4176EC7626B3 B193DE3B787FEC86E6CCB3F82F9812E9
dataMS <- dataMS[!dataMS$workerId %in% rmParticipant2,]
# Remove one participant with language impairments
rmParticipant3 <- unique(dataMS[dataMS$impairments=="Yes",]$workerId) #6C7F9A4676E026CD0E3056D8DA19E507
dataMS <- dataMS[!dataMS$workerId %in% rmParticipant3,]
# Detect participant whose median reactionTime is shorter than 2*SD below the mean of all MS
median_RT <- aggregate(dataMS$reactionTime, by=list(dataMS$workerId), median)
names(median_RT) <- c("workerId","median");dataMS <- merge(dataMS,median_RT,by="workerId")
cut <- mean(median_RT$median)-2*sd(median_RT$median)
# median_RT[!median_RT$median > cut,]$workerId # None detected!
# Remove one participant whose pattern of responses (SD) is below 2SD of the mean of all MS participants
SD <- aggregate(dataMS$enteredResponse, by=list(dataMS$workerId), sd)
cut <- mean(SD$x)-2*sd(SD$x)
rmParticipant4 <- SD[!SD$x > cut,]$Group.1 #B33F8EB38F232BB78667404E1D96025B
dataMS <- dataMS[!dataMS$workerId %in% rmParticipant4,]
# Check the total number of usable MS participants
# length(unique(dataMS$workerId)) #40

# Removing unusable NMS participants
dataNMS <- dataNotFiltered[dataNotFiltered$group=="NMS",]
# Remove one participant who did 436 items instead of 240 (length 4)
nbResponse <- aggregate(dataNMS$enteredResponse, by=list(dataNMS$workerId),  FUN=length)
rmParticipant5 <- nbResponse[!nbResponse$x %in% c(320,240),]$Group.1 # A7017E9F3DD1CC6778A3BC47958CA9B7
dataNMS <- dataNMS[!dataNMS$workerId %in% rmParticipant5,]
# Remove fourteen participants whose speakMaori or compMaori is at least (equal to or above) 3
rmParticipant6 <- unique(dataNMS[dataNMS$speakMaori >= 3 | dataNMS$compMaori >= 3,]$workerId)  # 04210C8909800F4041D775C8AE24720D 1E95A51880B3908528596F1CDC142382 1F90388DD2A6052379B3400E7E075B7B 2AABDB6A588C0A38D4D18DCCE2E411E0 3116B3E848662645051AE5AAA5D5E696 349856A9453BF2B1F7A026618E5F43D5 42564FDD3FE2B481B37DE8120E0BBB7C 5DBE9FB11F0E1FF4E5DA1149581576D0 90F93AC5E06AE66B73BFF3B46023FE3C A4B98DB8F5C9298E9FBF865FEB782486 B33F8EB38F232BB78667404E1D96025B C6243B5C4607EF24FD7F4D70B4CD0F41 C6C55B2283D35484E19BE5F4283F64D1 EB57DF86D41FFA1AD74E282B72D82856
dataNMS <- dataNMS[!dataNMS$workerId %in% rmParticipant6,]
# Remove ten participants who did not learn their English in NZ and have been living in their current location in NZ for less than ten years (duration == "short")
summaryNMSWorkerId <- unique(dataNMS[,c("workerId","firstLangCountry","place","duration")])
EngNotInNZ <- summaryNMSWorkerId[!summaryNMSWorkerId$firstLangCountry=="NZ",]
rmParticipant7 <- unique(EngNotInNZ[EngNotInNZ$duration=="short",]$workerId) # 21C1D24B57F6157A628A04DC5A519135 2AB61EA1EEBB7193A3A178D5B819225E 37A91EA9A4A92600B4ECEA239DC5D149 422774F5956A4A43258293A27B497995 4894A6F7674ECBFB304BCA4D9FB11B05 90FBA7311D5EB571D3C88425E4D371C0 E4E762BA8ADADA1CB0D943F8131F09B1 E862C480A8CE650797BE9D7309E52E15 FD2E69EFA7B5138BA6CC0BA31A10BFF0 FFA73BE62F1F21F94F35BC49A9B74838
dataNMS <- dataNMS[!dataNMS$workerId %in% rmParticipant7,]
#Remove one participant who has lived in Hawaii
rmParticipant8 <- unique(dataNMS[dataNMS$hawaii=="Yes",]$workerId) #6C3FBC26E49853CA97CFEB1DC3725C94
dataNMS <- dataNMS[!dataNMS$workerId %in% rmParticipant8,]
# Remove four participants who know any other Polynesian languages
rmParticipant9 <- unique(dataNMS[dataNMS$anyPolynesian=="Yes",]$workerId) # 84B5591229AA4FDE2C154B7873BCCC1E 905775C26C73AF3648FDAF216C1C836B B64C60A9F8121A22B1431FD012FE1BC1 EA6C551EB61BD1B4A3907ADCA5AC8A8C
dataNMS <- dataNMS[!dataNMS$workerId %in% rmParticipant9,]
# Remove three participants with language impairments
rmParticipant10 <- unique(dataNMS[dataNMS$impairments=="Yes",]$workerId) # 080658104D5BE0B7E5FDEDA8BCE32CED 504C5E7063935EB524BEC50FE180D24A CAA87C53B60F941BE5F4B3F4093B33D6
dataNMS <- dataNMS[!dataNMS$workerId %in% rmParticipant10,]
#Remove one participant whose median reactionTime is shorter than 2*SD below the mean of all NMS
median_RT <- aggregate(dataNMS$reactionTime, by=list(dataNMS$workerId), median)
names(median_RT) <- c("workerId","median");dataNMS <- merge(dataNMS,median_RT,by="workerId")
cut <- mean(median_RT$median)-2*sd(median_RT$median)
rmParticipant11 <- median_RT[!median_RT$median > cut,]$workerId # 0CC130754E535B995C330A62C70ABB29
dataNMS <- dataNMS[!dataNMS$workerId %in% rmParticipant11,]
# Remove four participants whose pattern of responses (SD) is below 2SD of the mean of all NMS participants
SD <- aggregate(dataNMS$enteredResponse, by=list(dataNMS$workerId), sd)
cut <- mean(SD$x)-2*sd(SD$x)
rmParticipant12 <- SD[!SD$x > cut,]$Group.1 # 23F8C303E5981DC46B3998C680DF6DFE 3ADCE0E04FE200BA98BC5C3015AF9092 90E5DE278473C80C5C9B1F50642DA2D5 CA1853385689F12642034089CDBFBAE3
dataNMS <- dataNMS[!dataNMS$workerId %in% rmParticipant12,]
# Check the total number of usable NMS participants
# length(unique(dataNMS$workerId)) #113

# Removing unusable US participants
dataUS <- dataNotFiltered[dataNotFiltered$group=="US",]
# Remove one participant who did 248 items instead of 240 (length 3)
nbResponse <- aggregate(dataUS$enteredResponse, by=list(dataUS$workerId),  FUN=length)
rmParticipant13 <- nbResponse[!nbResponse$x %in% c(320,240),]$Group.1 # A9692Y27LBXT9
dataUS <- dataUS[!dataUS$workerId %in% rmParticipant13,]
# Remove fourteen participants whose speakMaori or compMaori is above 0
rmParticipant14 <- unique(dataUS[dataUS$speakMaori > 0 | dataUS$compMaori > 0,]$workerId)  # A1AWHHBTJF1XCH A1L92Y6VBTRFP5 A1YAVTFZ9BULVV A28AX4H70DPKKK A2AMI7BVALOCJP A2DVRTL0JB0GT0 A2N0C5I6ZHNWV A2O5J0420C1UPC A2O762LZEZN3JC A4T642D9Z2UYC  AGVIHLR1DX7    AMKEGABQMZYIK  AMW2XLD9443OH  AWJUGWPCUGKEG 
dataUS <- dataUS[!dataUS$workerId %in% rmParticipant14,]
# Remove six participants whose maoriList is above 0
rmParticipant15 <- unique(dataUS[dataUS$maoriList > 0,]$workerId) # A19R7FSNDW6WRK A21KGSK2BDVWOA A2XOYCKJJ4PYZA A3ET9X8TXMGXDI A3V8C77FN4FLUA AP4FDDWBJW47O 
dataUS <- dataUS[!dataUS$workerId %in% rmParticipant15,]
# Remove one participant who did not learn English in the US
rmParticipant16 <- unique(dataUS[!dataUS$firstLangCountry=="US",]$workerId) # A33MBBGYASZPKN
dataUS <- dataUS[!dataUS$workerId %in% rmParticipant16,]
#Remove three participants who have been to Hawaii
rmParticipant17 <- unique(dataUS[dataUS$hawaii=="Yes",]$workerId) # A1QLYBRMIULAYC A3L0XKGZL9P2ZC ADY44EV3GCW85
dataUS <- dataUS[!dataUS$workerId %in% rmParticipant17,]
#Remove one participant with language impairments
rmParticipant18 <- unique(dataUS[dataUS$impairments=="Yes",]$workerId) # A1NUKP1UCT2R2R
dataUS <- dataUS[!dataUS$workerId %in% rmParticipant18,]
# Detect participant whose median reactionTime is shorter than 2*SD below the mean of all US
median_RT <- aggregate(dataUS$reactionTime, by=list(dataUS$workerId), median)
names(median_RT) <- c("workerId","median");dataUS <- merge(dataUS,median_RT,by="workerId")
cut <- mean(median_RT$median)-2*sd(median_RT$median)
# median_RT[!median_RT$median > cut,]$workerId # None detected!
# Remove one participant whose pattern of responses (SD) is below 2SD of the mean of all US participants
SD <- aggregate(dataUS$enteredResponse, by=list(dataUS$workerId), sd)
cut <- mean(SD$x)-2*sd(SD$x)
rmParticipant19 <- SD[!SD$x > cut,]$Group.1 # A1UHNDB2EQ8TK0
dataUS <- dataUS[!dataUS$workerId %in% rmParticipant19,]
# Check the total number of usable US participants
# length(unique(dataUS$workerId)) #94
dataExp2 <- rbind.fill(dataMS, dataNMS, dataUS)

3.2 Dataset structure

The data is structured as follows:

• workerId is the unique ID for each participant.
• enteredResponse is the wellformedness rating for each stimulus.
• reactionTime is the reaction time for each rating (second).
• word is the stimulus used for the rating.
• speakMaori is each participant’s answer to the question how well they can speak Māori (with a scale ranging from 0 to 5).
• compMaori is each participant’s answer to the question how well they can understand/read Māori (with a scale ranging from 0 to 5).
• maoriProf is the sum of quantified response for speakMaori and compMaori, which refers to the level of each participant’s Māori proficiency.
• age is the age group where each participant belongs to .
• gender is the gender of each participant.
• ethnicity is categorized into binary answers, either Māori (M) or non Māori (non M).
• education is each participant’s highest level of education.
• children is each participant’s answer to the question whether they have had any children who have attended preschool or primary school in New Zealand in the past five years.
• maoriList is each participant’s basic knowledge of Māori (with a scale ranging from 0 to 9).
• place is each participant’s current place of living (categorized into binary classification, either North or South Island in New Zealand).
• duration is each participant’s time duration of living in their current place (categorized into binary classification, long: > 10 years and short: =< 10 years).
• firstLang is each participant’s first language.
• firstLangCountry is the country where each participant learned their first language.
  
• anyOtherLangs is the information regarding any other languages each participant can speak.
• hawaii is the binary response to the question whether participants have lived in Hawaii.
• anyPolynesian is the binary response to the question whether participants know any Polynesian such as Hawaiian, Tahitian, Sāmoan, or Tongan.
• whichPolynesian is the information regarding participants’ knowledge of any Polynesian languages if they knew any.
• impairments is the answer to the question whether participants have a history of any speech or language impairments.
• maoriExpo is each participant’s level of exposure to Māori (with a scale ranging from 0 to 10).
• group is the classification of participants according to their fluency of Māori and/or exposure to Māori.
• nz is the binary response to the question whether participants have ever lived in New Zealand.
• median is the median of each participant’s reaction time.
• scoreDic is the dictionary phonotactic score normalized by the phonemic length of each stimulus.
• scoreRs is the running speech phonotactic score normalized by the phonemic length of each stimulus.
• scoreUnseg is the unsegmented running speech phonotactic score normalized by the phonemic length of each stimulus - 2.
• scoreDictNze is the score of the phonotactics derived from the dictionary of Māori words in New Zealand English.
• scoreSmall is the score of the phonotactics derived from NMS’ small active lexicon of Māori (comprising of 121 common loanwords that most NMS can identify as Māori words in our previous study plus 55 urban placenames).
• scoreFunc is the score of the phonotactics generalized over a list of function words.
• scoreEng is the score of the English phonotactics obtained from the English lexical database CELEX.
• scoreCV is the word shape score obtained from a trigram language model built by identifying each segment as a consonant, vowel or long vowel and calculating probabilities over sequences of those categories.
• scorePosLength is the position-length specific probability of phonemes obtained from a unigram language model.
• scoreDictNew
• scoreRsNew
• scoreUnsegNew
• scoreDictNoMacron is the dictionary phonotactic score without the distinction between short and long vowels marked with macrons.
• scoreRsNoMacron is the running speech phonotactic score without the distinction between short and long vowels marked with macrons.
• scoreUnsegNoMacron is the unsegmented running speech phonotactic score without the distinction between short and long vowels marked with macrons.

3.3 Participants’ basic Māori knowledge and proficiency

Fig.S5: Participants’ basic Māori knowledge and proficiency in Exp2.

3.4 Overview of participants’ profile

Fig.S6: Overview of participants’ profile in Exp2.

3.5 Overview of participants’ profile

Fig.S7: Overview of participants’ profile in Exp2.

3.6 Number of participants per length within each group

3.7 Modeling all groups’ well-formedness ratings with a linear mixed-effects model

# Model for Table.S4
dataExp2$group <- relevel(dataExp2$group, ref="NMS")
# modelTableS4 <- lmer(enteredResponse ~ c.(scoreDict)*group + c.(scoreRs)*group + c.(scoreUnseg)*group + (1 + c.(scoreDict) + c.(scoreRs) + c.(scoreUnseg)|workerId) + (1 + group|word), control=lmerControl(optimizer="bobyqa"), dataExp2)
# saveRDS(modelTableS4, file = "modelTableS4.rds")
modelTableS4 <- readRDS("./modelTableS4.rds")
kable(xtable(summary(modelTableS4)$coef), digits=3, caption="Table.S4: Modeling all groups' well-formedness ratings with a linear mixed-effects model. All numeric variables in this model are centered.")

3.8 Code for plotting Fig.2

Fig.2: Interaction between phonotactic scores and three groups. The range of phonotactic score is represented on the x-axis and the range of predicted rating is represented on the y-axis.

3.9 Post hoc exploration

# Example of R code to randomly sample 1000 words from the Māori dictionary
 N <- 1000; m <- 1000
 set.seed(1234)
 for(i in 1:N){
 randomWords <- sample(maoriDictionary, m)
 randomWordsTri <- orthoCoding(randomWords,g=c(3)) 
 randomWordsTri <- gsub("_"," ",randomWordsTri)
 outputFile <- paste0("randomization_", i, ".txt")
 write.table(randomWordsTri, outputFile, row.names = FALSE, col.names=FALSE, quote=FALSE)
 }  

# Example of R code to run linear mixed-effects models with 1000 samples
listLmerfit <- list()
for(i in 1:1000){
  lmerfit<- lmer(enteredResponse ~ c.(score) + (1 + c.(score)|workerId) + (1|word), data=files[[i]])
  listLmerfit[[i]] <- coef(summary(lmerfit))[2,4];message('Regression model of', i,'\n')
}
listLmerfitFinal <- unlist(listLmerfit)

3.9.2 Code for plotting Fig.3

Fig.3: Monte Carlo Simulations with 1,000 random samples

3.9.3 Density plot of t values for frequency-weighted random samples

Figure 6: Density plot of t values for frequency-weighted random samples

# The following predictors are added one by one to the linear mixed-effects model presented in Fig.2.
# (a) Phonotactics derived from the dictionary of Māori words in New Zealand English (Macalister, 2005)
# modelMāoriBorrowings <- lmer(enteredResponse ~ c.(scoreDictNze)*group + c.(scoreDict)*group + c.(scoreRs)*group + c.(scoreUnseg)*group + (1 + c.(scoreDictNze) + c.(scoreDict) + c.(scoreRs) + c.(scoreUnseg)|workerId) + (1+group|word), control=lmerControl(optimizer="bobyqa"), dataExp2)
# saveRDS(modelMāoriBorrowings, file = "modelMāoriBorrowings.rds")
modelMāoriBorrowings <- readRDS("./modelMāoriBorrowings.rds")
kable(xtable(summary(modelMāoriBorrowings)$coef), digits=3, caption="Modeling well-formedness ratings with a linear mixed-effects model including the phonotactics derived from the dictionary of Māori words in New Zealand English")

# (b) Phonotactics derived from NMS’ small active lexicon of Māori (comprising of 121 common loanwords that most NMS can identify as Māori words in our previous study plus 55 urban placenames)
# modelNMSLexicon <- lmer(enteredResponse ~ c.(scoreSmall)*group + c.(scoreDict)*group + c.(scoreRs)*group + c.(scoreUnseg)*group + (1 + c.(scoreSmall) + c.(scoreDict) + c.(scoreRs) + c.(scoreUnseg)|workerId) + (1+group|word), control=lmerControl(optimizer="bobyqa"), dataExp2)
# saveRDS(modelNMSLexicon, file = "modelNMSLexicon.rds")
modelNMSLexicon <- readRDS("./modelNMSLexicon.rds")
kable(xtable(summary(modelNMSLexicon)$coef), digits=3, caption="Modeling well-formedness ratings with a linear mixed-effects model including the phonotactics derived from NMS’ small active lexicon of Māori")

# (c) Phonotactics generalized over a list of function words
# modelFunction <- lmer(enteredResponse ~ c.(scoreFunc)*group + c.(scoreDict)*group + c.(scoreRs)*group + c.(scoreUnseg)*group + (1 + c.(scoreFunc) + c.(scoreDict) + c.(scoreRs) + c.(scoreUnseg)|workerId) + (1+group|word), control=lmerControl(optimizer="bobyqa"), dataExp2)
# saveRDS(modelFunction, file = "modelFunction.rds")
modelFunction <- readRDS("./modelFunction.rds")
kable(xtable(summary(modelFunction)$coef), digits=3, caption="Modeling well-formedness ratings with a linear mixed-effects model including the phonotactics generalized over a list of function words")

# (d) English phonotactics obtained from the English lexical database CELEX (Baayen, Piepen-brock & van H, 1993)
# modelEng <- lmer(enteredResponse ~ c.(scoreEng)*group + c.(scoreDict)*group + c.(scoreRs)*group + c.(scoreUnseg)*group + (1 + c.(scoreEng) + c.(scoreDict) + c.(scoreRs) + c.(scoreUnseg)|workerId) + (1+group|word), control=lmerControl(optimizer="bobyqa"), dataExp2)
# saveRDS(modelEng, file = "modelEng.rds")
modelEng <- readRDS("./modelEng.rds")
kable(xtable(summary(modelEng)$coef), digits=3, caption="Modeling well-formedness ratings with a linear mixed-effects model including the English phonotactics obtained from the English lexical database CELEX")

# (e) Word shape score obtained from a trigram language model built by identifying each segment as a consonant, vowel or long vowel and calculating probabilities over sequences of those categories
# modelWordshape <- lmer(enteredResponse ~ c.(scoreCV)*group + c.(scoreDict)*group + c.(scoreRs)*group + c.(scoreUnseg)*group + (1 + c.(scoreCV) + c.(scoreDict) + c.(scoreRs) + c.(scoreUnseg)|workerId) + (1+group|word), control=lmerControl(optimizer="bobyqa"), dataExp2)
# saveRDS(modelWordshape, file = "modelWordshape.rds")
modelWordshape <- readRDS("./modelWordshape.rds")
kable(xtable(summary(modelWordshape)$coef), digits=3, caption="Modeling well-formedness ratings with a linear mixed-effects model including word shape scores")

# (f) Position-length specific probabilities of phonemes obtained from a unigram language model
# modelPosLength <- lmer(enteredResponse ~ c.(scorePosLength)*group + c.(scoreDict)*group + c.(scoreRs)*group + c.(scoreUnseg)*group + (1 + c.(scorePosLength) + c.(scoreDict) + c.(scoreRs) + c.(scoreUnseg)|workerId) + (1+group|word), control=lmerControl(optimizer="bobyqa"), dataExp2)
# saveRDS(modelPosLength, file = "modelPosLength.rds")
modelPosLength <- readRDS("./modelPosLength.rds")
kable(xtable(summary(modelPosLength)$coef), digits=3, caption="Modeling well-formedness ratings with a linear mixed-effects model including position-length specific probabilities of phonemes")

# (g) Presence of macron(s)
# dataExp2$macron <- FALSE
# dataExp2[grepl("ā|ē|ī|ō|ū",dataExp2$word),]$macron <- TRUE
# modelMacron <- lmer(enteredResponse ~ c.(scoreDict)*group + c.(scoreRs)*group + c.(scoreUnseg)*group + macron*group + (1 + c.(scoreDict) + c.(scoreRs) + c.(scoreUnseg) + macron|workerId) + (1+group|word), control=lmerControl(optimizer="bobyqa"), dataExp2)
# saveRDS(modelMacron, file = "modelMacron.rds")
modelMacron <- readRDS("./modelMacron.rds")
kable(xtable(summary(modelMacron)$coef), digits=3, caption="Modeling well-formedness ratings with a linear mixed-effects model including the presence of macron(s)")

# (h) Presence of digraph(s) such as wh and ng
dataExp2$digraph <- FALSE
dataExp2[grepl("wh|ng",dataExp2$word),]$digraph <- TRUE
# modelDigraph <- lmer(enteredResponse ~ c.(scoreDict)*group + c.(scoreRs)*group + c.(scoreUnseg)*group + digraph*group + (1 + c.(scoreDict) + c.(scoreRs) + c.(scoreUnseg) + digraph|workerId) + (1+group|word), control=lmerControl(optimizer="bobyqa"), dataExp2)
# saveRDS(modelDigraph, file = "modelDigraph.rds")
modelDigraph <- readRDS("./modelDigraph.rds")
kable(xtable(summary(modelDigraph)$coef), digits=3, caption="Modeling well-formedness ratings with a linear mixed-effects model including the presence of digraph(s)")

# (i) Maximum length of vowel sequence
# Add the maximum length of vowel sequence
dataExp2$vSeq <- gsub("[aeiouāēīōū][aeiouāēīōū][aeiouāēīōū][aeiouāēīōū][aeiouāēīōū]","5", dataExp2$word)
dataExp2$vSeq <- gsub("[aeiouāēīōū][aeiouāēīōū][aeiouāēīōū][aeiouāēīōū]","4", dataExp2$vSeq)
dataExp2$vSeq <- gsub("[aeiouāēīōū][aeiouāēīōū][aeiouāēīōū]","3", dataExp2$vSeq)
dataExp2$vSeq <- gsub("[aeiouāēīōū][aeiouāēīōū]","2", dataExp2$vSeq)
dataExp2$vSeq <- gsub("[aeiouāēīōū]","1", dataExp2$vSeq)
dataExp2$vSeq <- gsub("[whkmrngpt]","", dataExp2$vSeq)
dataExp2$vSeq <- gsub(""," ",dataExp2$vSeq)
vSeqList <- strsplit(dataExp2$vSeq,split=" ");vSeqList1 <- sapply(vSeqList,max)
dataExp2$vSeq <- vSeqList1;dataExp2$vSeq <- as.numeric(dataExp2$vSeq)
# modelVSeq <- lmer(enteredResponse ~ c.(scoreDict)*group + c.(scoreRs)*group + c.(scoreUnseg)*group + c.(vSeq)*group + (1 + c.(scoreDict) + c.(scoreRs) + c.(scoreUnseg) + c.(vSeq)|workerId) + (1+group|word), control=lmerControl(optimizer="bobyqa"), dataExp2)
# saveRDS(modelVSeq, file = "modelVSeq.rds")
modelVSeq <- readRDS("./modelVSeq.rds")
kable(xtable(summary(modelVSeq)$coef), digits=3, caption="Modeling well-formedness ratings with a linear mixed-effects model including the maximum length of vowel sequence")

# (j) Type of first segment (consonant or vowel)
dataExp2$first <- gsub("[aeiouāēīōū]","v",dataExp2$word)
dataExp2$first <- gsub("[whkmrngpt]","c",dataExp2$first)
dataExp2$first <- substr(dataExp2$first,1,1)
dataExp2$first <- as.factor(dataExp2$first)
# modelFirst <- lmer(enteredResponse ~ c.(scoreDict)*group + c.(scoreRs)*group + c.(scoreUnseg)*group + first*group + (1 + c.(scoreDict) + c.(scoreRs) + c.(scoreUnseg) + first|workerId) + (1+group|word), control=lmerControl(optimizer="bobyqa"), dataExp2)
# saveRDS(modelFirst, file = "modelFirst.rds")
modelFirst <- readRDS("./modelFirst.rds")
kable(xtable(summary(modelFirst)$coef), digits=3, caption="Modeling well-formedness ratings with a linear mixed-effects model including the type of first segment")

# (k) New phonotactic scores
# modelNewScores <- lmer(enteredResponse ~ c.(scoreDictNew)*group + c.(scoreRsNew)*group + c.(scoreUnsegNew)*group + (1 + c.(scoreDictNew) + c.(scoreRsNew) + c.(scoreUnsegNew)|workerId) + (1+group|word), control=lmerControl(optimizer="bobyqa"), dataExp2)
# saveRDS(modelNewScores, file = "modelNewScores.rds")
modelNewScores <- readRDS("./modelNewScores.rds")
kable(xtable(summary(modelNewScores)$coef), digits=3, caption="Modeling well-formedness ratings with a linear mixed-effects model including the phonotactics....")

# (l) Phonotactic scores without the distinction between short and long vowels marked with macrons
# modelNoMacron <- lmer(enteredResponse ~ c.(scoreDictNoMacron)*group + c.(scoreRsNoMacron)*group + c.(scoreUnsegNoMacron)*group + (1 + c.(scoreDictNoMacron) + c.(scoreRsNoMacron) + c.(scoreUnsegNoMacron)|workerId) + (1+group|word), control=lmerControl(optimizer="bobyqa"), dataExp2)
# saveRDS(modelNoMacron, file = "modelNoMacron.rds")
modelNoMacron <- readRDS("./modelNoMacron.rds")
kable(xtable(summary(modelNoMacron)$coef), digits=3, caption="Modeling well-formedness ratings with a linear mixed-effects model with phonotactic scores without the distinction between short and long vowels marked with macrons")